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Let f n denote a kernel density estimator of a continuous density 
/ in d dimensions, bounded and positive. Let ^(t) be a positive con- 
tinuous function such that \\iSff \\ x < oo for some < /3 < 1/2. Under 
natural smoothness conditions, necessary and sufficient conditions for 



the sequence W 2 | ™ og n h d | II ^ ft) (fn(t) — Ef n (t))\\ x to be stochastically 
bounded and to converge a.s. to a constant are obtained. Also, the 
case of larger values of /3 is studied where a similar sequence with 
a different norming converges a.s. either to or to +oo, depending 
on convergence or divergence of a certain integral involving the tail 
probabilities of ^f(X). The results apply as well to some discontinuous 
not strictly positive densities. 



1. Introduction. Over forty years ago, Parzen (1962) studied basic prop- 
erties of kernel density estimators following their introduction by Rosenblatt 
(1956). Since then the kernel density estimator has become a classical object 
looked at by both statisticians and probabilists. For statisticians, it has been 
a canonical example of nonpar ametric curve estimators, which brought many 
important ideas from approximation theory and harmonic analysis into non- 
parametric statistics. Probabilists used the study of this estimator to test 
the strength of the methods from weak and strong convergence, empirical 
processes and probability in Banach spaces. In this paper, we consider a 
couple of problems about asymptotic behavior of kernel density estimators 
uniformly over all of R d that do not seem to have been considered before, 
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particularly in the 1980s, when the basic results on uniform a.s. convergence 
were obtained. 

The kernel density estimator f n of / corresponding to a sample of size n, 
a kernel K and a bandwidth h > is 



1.1 



fn(t) 



1 

nh d 



J2 K 



Xj-t 

h 



where Xi are i.i.d. with density /. To ensure its consistency, h is chosen 
to be a function h n of n such that h n — > and nh n — > oo as n — > oo. This 
is a biased estimator, but we will not deal with the bias; we will only be 
interested in the sup norm of the deviation of f n from its mean. 

Our starting point is the following well-known result due to Stute (1984): 



(1.2) 



lim 

n— >oo 



2 1 log h d \ 



fn - Ef n 



Vf 



\K\ 



a.s. 



where J is a compact parallellepiped with sides parallel to the axes, || • ||j 
means "sup over J," / is a uniformly continuous density which is bounded 
away from on J, and K is continuous and satisfies some additional as- 
sumptions [see, e.g., condition (K)]. Much later it was shown that 



(1.3) 



lim • 

n— >oo 



nh d 



log M\ 



Efn\ 



\K\ 



1/2 

oo 



a.s., 



where K satisfies condition (K) and / is uniformly continuous [Gine and 
Guillou (2002) for any d, and Deheuvels (2000) for d= 1; a weaker result of 
this type was obtained much earlier by Silverman (1978)]. In both results the 
bandsequence {h n } satisfies Stute's (1982) conditions. In fact, these results 
can be slightly extended as follows: if ^ is uniformly continuous and bounded 
on J, where J is either a bounded parallellepiped of R rf with sides parallel 
to the axes, or J = H d , then 



(1.4) 




nh d 



2|log/4l 



mWn(t)-Ef n (t))\l 



l^lbll*/ 



1/2, 



a.s., 



a result formulated in Deheuvels (2000) for d = 1 and which follows for any d 
from Einmahl and Mason (2000) and Gine and Guillou (2002) (with simple 
modifications in their proofs). Note that (1.4) contains (1.2) and (1.3). 

The first question on which we wish to shed some light is whether one 
can interpolate between the two results (1.2) and (1.3) by replacing J by H d 
and J" 1 / 2 by f~P for some < f3 < 1/2 in (1.2). A more general formulation 
of the same problem is whether unbounded functions \P are allowed in (1.4) 
when J = H d . 
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Notice that, in case / > over all of R d and limm-fui^^ f(x) = 0, (1.4) 
implies that only powers of (3 not exceeding 1/2 can lead to finite a.s. limits 
for the sequence 



(1.5) 



n 



hi 



2|log/4l 



fn — Ef n 



P 



n=l 



This is the case of classical norming, and in this case we find necessary 
and sufficient conditions (on the density / and on the bandsequence h n ) for 
(1.5) to be stochastically bounded (Theorem 2.1); in fact, Theorem 2.1 gives 
necessary and sufficient conditions for 



(1 - 5 '» W^ii»«</.m-*/.«>ii, 



n=l 



to be stochastically bounded, assuming ||*/^||oo < co for some (3 £ (0,1/2). 
This result further clarifies the role of the sequence of maximum terms 

n \ in the asymptotic behavior of (1.5') in prob- 
ability or in law. We also obtain a necessary and sufficient condition for 
(1.5') to converge a.s. to the constant ||.ff||2||\Ii r / 1 / 2 ||oo and show that if this 
condition is violated, then the sequence (1.5') is a.s. unbounded (Theorem 
2.6). 

A second question is that of determining the right norming constants 
in the sequences (1.5) or (1.5') for larger values of (3 in order to obtain 
convergence. In this case, we also give necessary and sufficient conditions 
for stochastic boundedness (Theorem 3.1) and for a.s. convergence of the 
sequences (Theorem 3.4). The almost sure limit is shown to be either or 
+oo, depending on convergence or divergence of a certain integral describing 
the tail behavior of $>(X). The situation in this case is somewhat similar 
to what is well known about weighted empirical processes; see Einmahl and 
Mason (1985a, b, 1988). 

We consider a slightly more general situation where / need not be strictly 
positive, however, we still require that, if Bf = {/ > 0}, then / be bounded 
away from zero on -B/H {\t\ < a} for all a > 0. Even this case requires unusual 
but somewhat natural smoothness conditions on /. More general situations 
seem to require a strengthening of the smoothness conditions, and we refrain 
here from considering them (see, however, Example 2.12). 

Assumptions and notation. We introduce here some notation and con- 
ditions that are used throughout the paper. 

For x = (xi, . . . , Xd) £ R d , we set \x\ := maxi<j<d \x%\. We assume that the 
kernel K satisfies the following condition: 
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(K) K > 0, K ^ 0, is a bounded measurable function with support contained 
in [—1/2, \/2] d which belongs to the linear span (the set of finite linear 
combinations) of functions k > satisfying the following property: the 
subgraph of k, {(s,u) : k(s) > u}, can be represented as a finite number 
of Boolean operations among sets of the form 

{(s,u) :p(s,u) > <p{u)}, 

where p is a polynomial on R rf x R and (p is an arbitrary real function. 

Conditions of a similar type were used, for example, in Koltchinskii and 
Sakhanenko (2000). 

In particular, the above property is satisfied if the subgraph of A; is a 
semialgebraic set in H d x R [see Dudley (1999), page 165]. If K{x) = <p(p(x)), 
p being a polynomial and (ft a rea l function of bounded variation, then K 
satisfies (K) [see Nolan and Pollard (1987)]. 

Condition (K) is mainly imposed because if K satisfies it, then the class 
of functions 

T = | K (jj^j : 1 G R ^ h > oj 

has covering numbers 

N(F,L 2 (P), \\K\\ L2{P) e) < (|y, < e< 1, 

for some A and v finite and positive and for all probability measures P. 
Indeed, for a fixed polynomial p, the family of sets 

{{(s, it) :p((s - t)/h, u) > (p(u)} :teH d ,h>0} 

is contained in the family of positivity sets of a finite-dimensional space 
of functions, and then the entropy bound follows by Theorems 4.2.1 and 
4.2.4 in Dudley (1999). The entropy bound will be crucial in the proofs 
below. Since the map (x,t,h) >—¥ (x — t)/h is jointly measurable and K is 
measurable, the class T is image admissible Suslin [Dudley (1999), page 186], 
and this implies that the measurability of the empirical process indexed by 
T [or even by {^(t)K((- — t)/h)} with ^ continuous] is as good as if the 
class were countable; that is, we can ignore measurability of the sup of the 
empirical process over g £ T [cf. Dudley (1999), Corollary 5.3.5 and Theorem 
5.3.6, or Pollard (1984), pages 195-197]. We set k := ||if ||oo (which is strictly 
positive). 

The following assumptions on the density / will be used repeatedly: 

(D.a) / is a bounded density on R d continuous on its positivity set Bf := 

{t G H d : f(t) > 0}, which is assumed to be open, and lim^oo sup| 4 | >a f(t) = 
0. 
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(D.b) For all 6 > 0, there exist c G (0,oo) and ho > such that, for all 
\y\ < ho and all x G Bf, x + y G 5/, 



(D.c) For all r > 0, 



lim sup 

x,y.f(x)>hr 
x+y£B f ,\y\<h 



f(x + y) . 



/(*) 



o. 



In particular, if log / is uniformly continuous on R d , then conditions (D.a)-(D.c) 
are satisfied (this is true, e.g., for the symmetric exponential density or for 
uniformly continuous nonvanishing densities with power tails). The above 
conditions are satisfied as well by normal and double exponential densities 
even though their logarithms are not uniformly continuous. Note also that 
(D.b) implies infa;e.B/,|x|<a fi x ) > f° r all a < oo such that Bf C\{\x\ < a} ^ 
0, in particular, a continuous density with bounded support does not satisfy 
(D.b). Similarly, a density that has an isolated zero where it is continuous 
does not satisfy condition (D.b) either. In fact, Example 2.12 shows that, 
for such a density, the stochastic boundedness of the sequence (1.5) depends 
on the local behavior of the density at its zero points, and not only on the 
tails of the random variable f~@(X), as is the case under condition (D.b) 
(see Theorem 2.1). On the other hand, the exponential density does satisfy 
(D.a)-(D.c). 

Conditions (D.b) and (D.c) on / are not found in Stute's result (1982, 
1984) or in Einmahl and Mason (2000) because they consider bounded in- 
tervals with / bounded away from zero on them, and they are not found 
either in Gine and Guillou (2002) since there is no division by a power of / 
in their result. These conditions seem natural for the results that will follow 
and we will indicate below that conditions of this type are indeed needed; 
see Example 2.11. 

We assume that the weight function satisfies the following conditions 
that resemble the above conditions on the density: 

( W.a) VP : Bf i— > R + is a positive continuous function on Bf. 
(W.b) For all 5 > 0, there exist c G (0, oo) and ho > such that, for all 
\y\ < ho and all x G Bf, x + y G Bf, 



^ W <!^!<^). 



(W.c) For all r > 0, 



lim sup 

h ^°x,y: q>{x)<h- r 
x+y&B f ,\y\<h 



$(x + y) 



0. 



G 
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In particular, by (W.b), ^ is bounded on bounded subsets of Bf, but 
may be unbounded if Bf is unbounded. 

We also need the following conditions that establish a relationship between 
/ and ^: 

(WD. a)/? ||/^||^,oo := sup tgB/ \i&(t)fP(t)\ < 00, where is a positive num- 
ber. 

(WD.b) For all r > 0, 



lim sup 

ft_>0 x,y: t>(x)<h- r 
x ,x+t&B t ,\y\<h 



f{x + y) 



Six) 



0. 



Note that (WD.a)^ and (WD.b) imply (D.c): if ||*/^|| B/ < c and /(i) > 
h r , then *(t) <c/i" r/3 . 

Also, if \& = /~ " (which is our main example), then (WD. a) ^3 is satis- 
fied and the set of conditions (D.a)-(D.c) is equivalent to (W.a)-(W.c) and 
(WD.b). 

Regarding the window sizes, the assumptions are: 

(HI) ht, t> 1, is monotonicahy decreasing to and th% is a strictly increas- 
ing function diverging to infinity as t — > 00, and 

(H2) hf is regularly varying at infinity with exponent —a for some a £ (0, 1); 
in particular, there exist < 770 < rji < 1 such that 



lim sup t V0 hf = and lim inf t m hf 



OO. 



Condition (H2) is quite restrictive compared to the bandsequence assump- 
tions in Stute (1982): besides the extra regularity, we do not allow h t to get 
too close to the extremes 1/t or 1/logi, and in particular, | log /it | is com- 
parable to logi, t > 1. If we set 



X t = Vthf\logh t \, 

then, under (HI) and (H2), the function At is strictly increasing and is 
regularly varying with exponent larger than 0. This property of At is used 
throughout Section 2. 

Our results rely on the by now classical theorem of Stute (1984) about 
the a.s. behavior of the uniform deviation of the kernel density estimator 
over compact intervals, suitably modified. The version of his theorem we 
need is a reformulation along the lines of Deheuvels (2000) of Proposition 
3.1 in Gine and Guillou (2002), which in turn is adapted from Einmahl and 
Mason (2000). 
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Proposition 1.1. Let f be a density on H d , continuous on an open 
set containing D a := {t: \t\ < a,f(t) > a" 1 }, for some < a < oo. Let ^> be 
a strictly positive function, continuous on an open set containing D a . Then 



(!-6) Jim J^|^||*(t)(/n-^/n.)(t)b a = ||K|| 2 ||^/ 1/2 ||D a a.s. 



n— >oo 



We omit the proof as it coincides with the proof of the abovementioned 
proposition, except for obvious changes. 

Proposition 1.1 applies to / satisfying (D.a) and (D.b) and satisfying 
(W.a) and (W.b). 

Without further mentioning, all the results we state in this paper beyond 
this point assume conditions (K), (HI), (H2), (D.a)-(D.c), (W.a)-(W.c), 
(WD.b) and (WD.a)^ for some (3. The number (3 is to be specified at each 
instance. We will refer to these assumptions as the "usual hypotheses." 

Finally, we introduce the following notation, which will be used through- 
out: for any function g defined on Bf, we set 

(1.7) llslkoo — sup \g(t)V(t)\. 

teB f 

2. The classical norming case. The following theorem describes the stochas- 
tic boundedness behavior of the sequence (2.1). It shows in particular that no 
interpolation between (1.2) and (1.3) works for all strictly positive, bounded, 
continuous densities, and that when it works, it does not work for all the 
range of possible bandsequences. In what follows, X is a random variable 
with density /. 

Theorem 2.1. Assume the usual hypotheses, with condition (WD. a) ,3 
holding for some (5 G (0, 1/2), and, moreover, that either Bf = R d or K{Q) = 
|| K I Ioq. Then the sequence 



n=l 



is stochastically bounded if and only if 

(2.2) limsuptPr{*(X) > (thf | log h t |) 1/2 } < 00. 

t— >oo 

Moreover, under condition (2.2), the sequence 



■\\fn-Ef n \\ 



(2.3) 



2|log/4l 

/ max WW )v(\\Kh\\f 1/2 h,o°) 



pnhi\\oghi 
converges to zero in probability. 



n=l 
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Proof. We will use the notation X t = (th d \ log/ijl) 1 / 2 . As mentioned 
above, conditions (HI) and (H2) imply that At is regularly varying with 
strictly positive exponent. Note that, by regular variation, condition (2.2) is 
equivalent to 

(2.4) limsupiPr{^(X) > cX t } < oo 

for any < c < oo. By Montgomery-Smith's (1993) maximal inequality [see, 
e.g., de la Peha and Gine (1999), page 6] the stochastic boundedness of the 
sequence (2.1) implies that of the sequence 

/ \\K((Xi - t)/h n ) - EK((X - t)/h n )\\ %O0 \ 
max 



l<i<n X n 



Then, since, for all t, letting u = (u\, . . . ,Ud), 

EK{(X - t)/h n ) = h d n f 1/2 ■ ■ ■ f 1/2 K{u)f{h n u + t)d Ul --- du d 

J-l/2 J-l/2 
<^||^||l||/||oo->0, 

taking t = X\ — rh n , . . . , X n — rh n for r G H d satisfying K(t) > 0, we obtain 
that the sequence 

^(Xi-Th n )I{Xi-Th n €B f ) 



max 



l<i<n A n 

is stochastically bounded. We take r = if K(0) / 0. Now, if K(0) / 0, then 
Xi — rh n = X{ G Bj a.s., and if Bf = H d , then obviously Xi — rh n G Bf, so 
that, in either case, Pr{X — rh n G Bf} = 1. Thus, the sequence 

■$(Xi-Th n 



max 

l<i<n X n 

is stochastically bounded. In particular, if r = 0, 

max — 

l<i<n A n 

is stochastically bounded, proving condition (2.4) in this case (as, if £j are 
independent, Pr{ max |&| >c}>£Pr{|&| > c }/(! + £ Pr{l6l >c}).lfr^0 
but Bf = R d , given e > 0, let M be such that 

supPr^ max v — >M\<e. 

n (!<«<« A n J 

If ty{Xi — Th n ) < MX n , then by regular variation there exists r > such 
that ^(Xi — rh n ) < h~ r (at least for all n large enough), and we can apply 
condition (W.c) to conclude that there exists c > 1 such that, for all n large 
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enough (independent of Xj), ^pQ) < c^pQ — rh n ). Then, for these values 
of n, we obtain 

Pr( max > C M X < e. 

[l<i<n X n J 

Therefore, in this case, the sequence {maxK^n \l/pfj)/A n } is also stochas- 
tically bounded, proving (2.4). 

For the converse we note first that Proposition 1.1 takes care of the sup 
over D a for any a > 0. 

Next, we observe that the centering in (2.1) can be ignored for a certain 
range of t's. Let e n — > and < S < 1 — 0. Choose r > such that 

h r(l-P-S) 

0. 



{nhi)- l Xn 

Then there exist c < +oo and no < oo such that, by (D.b) and (D.c), for any 
t G Bf and n > n , 

n®(t)EK((X - t)/h n ) 

sup f(t + h n u) 

|«|<l/2 
t+h n u£Bf 

Since, by condition (WD.a)/3, ^(t) < cf~@(t), the last summand tends to 
uniformly in t (as j3 < 1 — 5). The sup of the first summand over all t such 
that V(t)f(t) < e l ~P {(nh^Y^logh^) 1 ! 2 tends to as well. Thus, we can 
ignore the centering Ef n (t) for all t S Bf such that 



(2.5) ¥(<)/(*) < 



1-/3/ I iu fe 



for any sequence e n — > 0. We take e n = 1/logn. 

In the rest of the proof, we consider the sup of \ty(t)(f n — Ef n )(t)\ over 
several regions. 

First, we consider the regions 

(2.6) A n :={teB f :^{t)>c? n X n } 

for the sequence c n = (X n \ ogn / 'X n ) 1 ^ which tends to infinity because Aj is 
regularly varying with positive exponent. Actually, if r/ > is the exponent 
of regular variation of At, the representation formula for regularly varying 
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functions [e.g., Feller (1971), page 282] gives that, for every < e < r\ and 
c > 1 , there exists no < oo such that 

(2.7) - (log n)^ e < < c(log n)"+ £ 

c X n 

for all n > no- Then, since (3 < 1/2 < 1 — (3, for a suitable 5 > and all n 
large enough, we have 

CnAy /3 >n^- 5 >n"/( 1 -«+ 5 



> W A n \V(i-/3) i / \ i/(2(i-/3)) 



e n \|log/in|/ e n \|log^ 
This yields, for all t E A n , 

T , , 1 / n/# \/V(2(i-fl) 

£ p v|iog/i n |/ 

and, using condition (WD.a)^ (which without loss of generality can be writ- 
ten as H/^H^oo < 1), we get 



/(*) < £r 



loghnW 1 ^ 1 -^ 



11 



hi 



This implies (2.5) for all t E A n , since *(*)/(*) < H^H^oo/ 1- ^*) < f l ~ P {t) 
[again due to (WD.a)^]. Therefore, 

nV(t)EK((X-t)/h n ) 
sup ► U, 

showing that we can ignore the centering Ef n on the region A n . For any 
point a = (ai, . . . , cy) E R d and positive number p, we set 

J (a; p) := [ai - p/2, ai + p/2] x • ■ • x [a d - p/2, a d + p/2] f]B f . 

Then, discarding the centering, 

sup mY.umx t - t) , K ) s sup « , w £ /w 6 J(t;M) . 

Now we divide A n into two parts: 

A ntl :={t£B f :^(t)>h- r } and A n , 2 := {t E % : c^A„ < < h~ r }, 
— r(l— S) r 

where r is such that /i n > c£A n for some 5 > and all n. It follows 
from condition (W.b) that t E A n i and s E J(t;h n ) imply that there are c 
and no such that 



> ctf- fl (t) 
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for all n > no, so that VP(s) > c^I /1_5 (t) > ch n rlyl . Hence, for the same 
values of n and some C < oo, we have 

p J sup * W SU *g* - *VM > A < nPr{ , w > 

C 

<; ►O. 

log re 

It follows from condition (W.c) that t £ A ni 2 and s £ J(t;h n ) D Bf imply 
that there are c and no such that ^f(s) > cf^X n /c, for all re > no- Hence, for 
these values of re we have 

P r sup mT.umx.-t) ik) > i < t > ^ < c ^ 

UeA n , 2 -^n J I c J log n 

for some C < oo by (2.4). The last two limits imply that 

(2.8) hm sup = m pr. 

n ^°°teA n A n 



Now we consider the regions 



(2.9) 



B„:={tGS/:/(t)*(t)<4-^ 



log^n|\ 1/2 



nh* 

^(t)<4(nhi\logK\) 1/2 



and notice that in these regions we can also ignore the centering [by (2.5)]. 
Our goal is to show that 

*(i)ELl#(PQ-*)Mn) 



sup 

tes„ A r 



n=l 



is stochastically bounded under condition (2.2) and that, moreover, if either 
B f = K d or K(0) = k, then also 

(2.10) sup = k max — h o p (l). 

t£B n Ki \<i<n \ n 

As above, 

*(t)Eg=i *)//*») ^ A 

sup < sup — — 2^-'v- x i G J{t;h n )), 

teB„ *n teB„ A n i=1 



and we set 



Z n := sup-^^/^E J(t;h n )). 
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For j = 1, . . . , n, set 

B n ,j '■= B n n J (X,- ; /i n ) . 
If t ^ Uj=i Bnj, then Z n = 0. Hence, we have 

Z n = max sup — — Vlfte J(i;/i„)). 

i<3<n teBnJ A n 

By conditions (W.c) and (WD.b), t G -B n j implies that 

ff(t) < c*{Xj) 

and also 

log U\ 1/2 



for any c > 1, provided that n is large enough. 
Set 

Ij = Inj ■= rh{Xj) < cc^X n , fiX^iXj) < ce 1 -" 
Then 

Zy^ \ LLlaX. 

< max — - — - — h max 



By condition (2.2), the first term in the above bound is the general term 
of a stochastically bounded sequence. We now show that the second term 
tends to zero in probability. To handle this term, let Pj denote conditional 
expectation given Xj and set 

Pj-Pj^X-X^Khn}. 

It follows from condition (D.a) that 

2 d c- 1 h d J(X j )<p j <2 d ch d J(X j ) 

(provided that Ij = 1). A standard bound on binomial probabilities [e.g., 
Gine and Zinn (1984), page 958] shows that 



^ ~ £ 

(n - \)epjV{Xj)\ (eAn/*(Jfi))vi 
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Using the bound on pj, this probability can be further bounded by 
( 2 d ecnh d f(X j )V{X j ) \ (^„/*(^-))vi 

We can and do assume that Ij = 1 (otherwise the conditional probability in 
question is 0). Then 



f{X 3 )^{X 3 )<ce' n 



^/iiog/gy/ 2 



n 



hi 



and we have 



2 d ecnh d n f{X 3 )t>{Xj) ^ C^hjej-^l \ogh n \/nh d n ) l l 2 _ 

A„e (n^|log/i n |)V2 e £ 

for some C\ < oo (and all n large enough). Note also that 

2 d ecnh d l f(X j )^(X j ) _ C ( nh d X 1 / 2 



X n e e \|log/ir 

C is a finite 

yields 



C E 1 ~ lS -1 / 

where C is a finite positive cosntant. For large n, — ^-z* — < e /£ , which 



A n n /C/ X 1 / 2 



Let 



and let J 2 := Ij — ij. Then we have 



rr< max — > e 

U<i<« A n 



n r x i n n / r7/7 d \ x / 2 



.7 

= :(/) + (//). 
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Then, using the definition of ij and ij, we get 

f 31ogn 1 _ 2 
(J) < nexpj ~\~ ) =n 



and 



eV|log/in|/ (n^llog/tnl)^-^)/^) I v ; -V31ogn 

Now, since is regularly varying with a strictly positive exponent, the rep- 
resentation theorem for regularly varying functions gives that A n /(31ogn) > 
cA n /(i ogn )7 for some 7 > 0, c> and all n large enough [see (2.7)]. Hence, 
by (2.4), there exists C > such that, for these values of n, 



E |log/l n | 1 /2+(l-/3)/(2/3) ( n/i d)(l-/3)/(2^)-l/2- 

By (HI) and (H2), this is at most of the order of logarithmic factors times 
n (i-r?i)[i/2-(i-/3)/(2^)] ) a negative power of n because < /3 < 1/2. Thus, 
(//) also tends to zero. Since both (/) and (II) tend to 0, we have 

pf ^^(^)El<Kn,^/(l^-^l<^) ^ 1 n 

Fr< max — — > as n — > 00. 

U<i<™ A n J 

This implies [see bound (2.11)] that 

(2.12) sup v " ^<KZ n <cKinax v y +o p (l), 

(6B„ A n l<i<n \ n 

for any c > 1. The stochastic boundedness of 



sup 

t£B n *n 



follows immediately from this inequality and condition (2.2). 

To bound the supremum from below, choose r such that K(t) > k — 5 (for 
a small 5) with the understanding that if K(0) = k, then we choose r = 0, 

f- 



so that either r = or Br = R d . Then 



tG_B„ A„ l<t<n A n 

Hence, in view of this and the two-sided bound for nZ n immediately above, 
to establish (2.10), it is enough to show that 



, 01 ^ *(AWfr w )/ Bn (AWfe n ) *(Xj) 
(2.13) max = max — \-o p (l). 

l<i<n \ n l<i<n \ n 
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Since condition (W.c) implies that, for any c > 1 and for large enough n, 

_! V(Xi - Th n ) 

(assuming that AQ — rh n € B n ), taking c arbitrarily close to 1 reduces the 
proof of (2.13) to showing that 

max = max — h o p (l), 

or, put in another way, (2.13) will be proved if we show that 

(2.13 ) max — >0 m pr. 

l<i<n \ n 

B n naturally decomposes into the union of three regions and we look sep- 
arately at each of them. If Bf = R d , then /gcpQ — rh n ) = 0, and if r = 0, 
then this indicator is a.s., so that, in either case, 

^(Xi)I B o(Xi-Th n ) 

max i- ► a.s. 

l<i<n \ n 

Next, we consider 

pJ mg »W f (»W-r*.)S<A.) > . 

(l<i<n A n 

<nPr{tf(X-rfc n )>c£A n } 

< nPr{c^A n < V(X - rh n ) < C} + nPr{*(X - rh n ) > h~ r }. 

Using condition (W.c), we get (for any c> 1) 

nPr{c^A n < *(X - rh n ) < h~ r } < nPr{V(X) > C 1 ^} 0. 

Similarly, using condition (W.b) (recall that X — rh n £ with probability 
1), 



Pr{* (X - rh n ) > K r } < nPr{^(X) > c -1 /i~ r / (1+a) } 



for some c > and <5 > 0. Assuming that r is large enough (sothat/ l - r/(1+<5) > 
c^A n ), we then conclude that 

nPr{^(X - rh n ) > K r } -> 

and, hence, 

(l<j<n A n 
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Before considering the last piece of B!^, we note that, since f^(t)^(t) < 1 
for all t, if moreover f(u)^(u) > L, then / 1_ ^(-u) > L and consequently 
^(u) < f~@(u) < L~P/( l ~@\ an observation that we will use several times 
below. This observation and condition (W.c) give 

^{Xi)I{f{Xj - ThnYHXj - rh n ) > 4-^(1 logKl/jnhj)) 1 / 2 ) 
max 

l<i<n \ n 

ViXiVWXi - rh n ) < ce-P{nhi/\ log h n \y/^-W) 



< max 



l<i<n \ n 



<ce~ p 



nhj y/^-m 1 

log /in I J (nhZ\logh n \y/* 
c 

~ ^|log/l n | 1 /2+/3/(2(l-/3))( n / l d)l/2-/3/(2(l-/3))- 

Now, since /3 < 1/2 < 1 — /3 and nh^ > n 1 " 711 [by (H2)], whereas e n = 1/logn 
and I \ogh n \ is comparable to logn, it follows that the above bound is dom- 
inated by a negative power of n so that, in particular, it tends to zero. This 
and the previous two limits conclude the proof of (2.13') and hence of (2.10). 

Finally, we consider the sup over the remaining set of t's. For a large, 
fixed, just as above, set 

(2.14) C n = C n , a := {t G D c a n Bf : /(t)tt(t) > e^fl \ogh n \/nh d n ) l l% 

where e n is as defined in the previous paragraph. In this range the centering 
cannot be ignored. We will apply an estimate for the expected supremum 
of the empirical process over bounded Vapnik-Cervonenkis type classes of 
functions [Gine and Guillou (2001), inequality (2.1) and Talagrand (1994), 
for classes of sets; see also Einmahl and Mason (2000) for a similar inequal- 
ity]: if a class of functions T is measurable (in particular, if it is image 
admissible Suslin) and satisfies 

(2.15) N(T,L 2 (Q),e\\F\\ 00 )<(^J, 0<e<l, 

for some v > 1, A > "iyfe finite and all finite probability measures Q, where 
F is a measurable envelope for the class then 



(2.16) E\\n(P n - P)y < C (^vW^y log ^ + vUlog ^ 

where a and U are any numbers satisfying < a < U and 

(2.17) a 2 > supVarp(g), U > sup ||s||oo, 
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and C is a universal constant. [In Gine and Guillou (2001), condition (2.15) 
has ||.F|| l 2 (Q) instead of H-FHooj but it can be easily checked that their proof 
works as well under condition (2.15).] As mentioned immediately below the 
statement of condition (K), there exist A and v finite such that 

N({K((--t)/h n ):teK},L 2 (Q),e)<(—) V , 0<e<l, 



for all h n > and all probability measures Q on R. Now, the class of func- 
tions 

F n :={*(t)K((--t)/h n ):t€C n } 

is contained in 

g n := {uK{{- - t)/h n ):t£ R,0 < u < U n }, 

where 

k / nh d \/3/(2(i-/5t)) 

(2 ' 18> ^i(nSb) 

[recall that, as observed above, under condition (WD.a)^, > a implies 
* < qT /3 /( 1 -0)]. Therefore, since the L 2 (Q) distance between uK((- —t)/h n ) 
and vK{{- — s)/h n ) is dominated by k\u — v\ + U n \\K{{- — t)/h n ) — K((- — 
s)//i n )||^ 2 (Q), it follows by taking optimal coverings of [0,C/ n ] with respect 
to the Euclidean distance, and of T n with respect to the L 2 (Q) distance, 
that the entropy bound 

(2.19) N(f n ,L 2 (Q),eU n )< I — ) , 0<e<l, 

holds for all probability measures Q and all n large enough. The class J- n 
is also image admissible Suslin since the map (x,t) \— > ty(t)K((x — t)/h n ) is 
measurable. So, inequality (2.16) applies to it. We can take U = U n as defined 
in (2.18). Next we estimate a\. It follows from a previous observation and 
from regular variation that, on C n , we have both f >h r n and \£ < h r n for 
some r and all n large enough. Then, (D.c) and (W.c) give that there exist 
c, C, no < oo independent of a such that, for all n > no and all t £ C n = C n>a , 

^ 2 (t)EK 2 ((X - t)/h n ) < cE(K 2 ((X - t)/h n )* 2 (X)) 

= chi \ K 2 (u)^ 2 (t + h n u)f(t + h n u)du 

J\u\<l/2 
t+h n u£Bf 

<ch d jK\\ 2 \\fV 2 \\ D c nBi 
<C f /4(||/* 2 || iJ c nfl/ Vn- 1 ). 
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So, we can take 



a 2 n = Ch d n (\\f^ 2 \\ D c nBf y 



n 



The constant A = A n must be taken to be (2Ak) V (3^/e), where A is the 
constant in (2.15) for the class consisting of translations and dilations of K. 
In particular, since by (H2) | \ogh n \ is comparable to log re, we have 

A n U n n , | 
log < c\ \ogh n \ 

for some constant c < oo independent of n. So, inequality (2.16) applied to 
T n gives 

vP(i) E?=i " t)/K) ~ EK((X - t)/h n )) 



E sup 

teC n 



A, 



C 
< — 

An 



A„(||/ 1/ ^||D S n B/ Vre^ 1 / 2 ) + - 



h d \ (9/(2(1-/8)) 



log /in I 



eg V I log ^ 

for a constant C independent of n, for all sufficiently large n. We should 
note that the numerical constants in the above inequalities are not only 
independent of re, but they are independent of a as well. Since (3 < 1/2 and 
therefore (3/(1 — f3) < 1, and since, by (D.a) and (WD.a)^, 



r 1/2-/3 1 



DgnBf 



1/2-/3 

Dsns 







as a — > oo, 



we obtain 



lim lim sup -E sup 
(2.20) a ^°° teC n , 



*(*) E?=l(^((^ " *)/^n) " EK({X - t)/h n )) 



A. 



< lim C\\f/ 2 nD^B f =0. 
Now, the theorem follows from (1.6), (2.8), (2.10) and (2.20). □ 



Now we make two comments on the assumptions. 



Remark 2.2. The assumption "B/ = R d or A'(0) = \\K W^" has been 
imposed because in general we may not have X — rh n G Bj with small 
enough probability, as nPr{X — rh n ^ Bf} could well be of the order of 
nh n — > oo. Now, this condition has been used in full only in the proof of 
(2.3). Proving that tightness of the sequence (2.1) implies condition (2.2) 
has only required Bf = R rf or K(0) > 0, whereas proving that condition 
(2.2) implies tightness of the sequence (2.1) does not require any hypothesis 
of this type. 



The above proof justifies, a posteriori, having taken (3 < 1/2: 
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Corollary 2.3. Assume (K), (HI), (H2), (D.a)-(D.c) and B f = K d . 
Then the sequence 



n 



hi 



21 log 



fn — Ef n 



Vf 



n=l 

[which coincides with (2.1) for ^ = f^ 1 ^ 2 ] is not stochastically bounded. 

Proof. By the first part of the previous proof, if (2.1) with f = f' 1 / 2 
is tight, then there is C > such that 

" Pr {/(W >A «} SC 

Since / takes all the values between and ||/||oo ? f° r n large enough there 
is x n in R d such that f(x n ) = 1/(2A^). Then, by condition (D.c), there is a 

—2/r 

subset D n containing x n and of Lebesgue measure larger than A n , where 
— an d f{x) > 1/(4A^), and therefore, if we take r > rji/d with r/i 
as in condition (H2), 

contradiction. □ 

Theorem 2.1 has the following obvious corollary regarding convergence in 
distribution: 

Corollary 2.4. Under the assumptions in Theorem 2.1, the sequence 
(2.1) converges in distribution if and only if the sequence of maxima, 

max 



l<i<n X n 

converges in distribution. Then, if Z is a random variable with distribution 
the limit of this last sequence, we have 



Xhr^d\\fn ~ Ef n \k,oc ^ (\\K\\ooZ) V (||K|| 2 ||/ 1/2 |k,oo). 

V 21og/i„ 

Next we consider the a.s. counterpart to Theorem 2.1. The following 
proposition will help. It is perhaps relevant to recall first a well-known fact, 
whose proof we omit as it is similar to a classical result of Feller [e.g., Lemma 
3.2.4, Corollary 3.2.3 and Theorem 3.2.5 in Stout (1974)]. 
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Lemma 2.5. Let Vi be i.i.d. real random variables and let {c(n)} be 
a nondecreasing sequence, regularly varying with strictly positive exponent. 
Then, 

either lim sup max ^ =00 a.s. or lim max ■ ^ =0 a.s. 
n^oo l<i<n C{n) ra->oo l<j<n c (n) 

And this happens according to whether 

J2~P*{\V n \ >Cc{n)} = 00 or ^Pr{|Fj > Cc(n)} < 00 

n n 

for some (or, equivalently, all) C > 0. 

Proposition 2.6. Assume that conditions (D.a), (W.a)-(W.c) and (WD.a)^, 
(WD.b) hold for some j3 > and that, moreover, either Bf = H d or K(0) > 
0. Let c(n) /* 00 be a regularly varying function of n. Assume 



T2=i(K((Xi - t)/K) - ek((x - t)/h n )) 



c(n) 



< 00 a.s. 

'J, 00 



(2.21) limsup 

n 

Then, 

(2,2) ? P r (ig) >C }<» 

for all < C < 00 or, what turns out to be the same by Lemma 2.5, 
(2.22') lim max -. ' = a.s. 

n— >oo l<i<n c(n) 

Proof. The proof is standard, but we give it here for completeness. 
First we note that if (2.21) holds and c(n) /* 00 is regularly varying, then 
c(n) has necessarily positive exponent, which by Lemma 2.5 gives the equiv- 
alence between (2.22) and (2.22'). This follows because, by (2.21), there is 
t with *(f) / and f(t) / such that the sequence J2i=i^(t)( K {^h^) ~ 
EK '(^-^)) / c(n) , n G N, is tight, which, by boundedness and finite support 
of K, implies that the sequence of its second moments is uniformly bounded, 
thus, that the sequence n/i^/c 2 (n) is bounded; hence, since by (H2) nh d is 
regularly varying with strictly positive exponent and c(n) is regularly vary- 
ing, it follows that the exponent of c(n) is strictly positive as well. 

Let {Xl} be an independent copy of {^Q}. We can symmetrize in (2.21) 
and still have the limsup finite. By continuity of ^ on Bf, there is n(u>) < 00 
a.s. such that, for all n > n(uj), 



K({X(u) - t)/h n ) - K{(X'{co) - t)/h n ) 



c(n) 



k(^{X) + ^{X') + 1) 
*,oo ~ c(n) 
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This tends to zero and therefore the limsup in (2.21) is a.s. constant by the 
zero-one law. Hence, we have 



Pr-| sup 



for some c < oo. Set 



£r=i 9(t)(K((Xi ~ t)/K) - K{{X> - t)/h n )) 



c(n) 



>c\->0 



as k — > oo 



H ( XX >) := *W( K « X ~ t }IM ~ K ^ X ' ~ *)/M) , 
n ' ' c(n) 

and, for k 6 N, 

2i,fc = (Hk^XijX'i), Hk+i(Xi,Xi), . . . , fT^^^Xj-), . . . ) 
if i < k, and 

^ijfe = (0, - r ). , 0, H k+r (X k+r , X' k+r ), Hk+ r+ i(Xk+r, X k+r ) ... 

for i = k + r, r = 1, Then, the above sup over n > k is simply 

oo 

8=1 

where |||(xi(£), . . . ,x n (t), . . .)||| = sup n ||:En(*)||B/- The random vectors Z i)k are 
independent and symmetric, and we can apply Levy's inequality to get that 

Pr{sup|||Z ifc ||| > 2c 1 -^0 
as k — > oo. By independence, this implies that 

oo 

^Pr{|||Z lifc |||>2c}^0 

8=1 

as k — ► oo. Let r = if K(0) > and otherwise let |r| < 1 be such that 
K{t) > 0. Then, 

\\H m {x, x')\\ B} > ^ {x ~^ mT) i{\x - X>\ > h n ) 



c(n) 



for some c > 0, and we get that 



\Z l:k \\\ = sup H m (Xi,Xi)>?p£l 

m>iVk C{1 V k) 



I{\X l -X' i \>h iyk ) 



when t = and 

|||^ fc ||| = sup H m (X h Xi) > ^tei^Lffi ~ _ X '\ > ft. vfc ) 
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when Bf = R d . The case r = is easier to handle, so we will complete 
the proof only for the second case. In this case, since Pr'{|X — X'\ > hi} > 
1 — ||/||oo^f) the previous inequality yields 

± P rm, tl > 2c} > £(1 " W~>4)K{ SMm yy X ~ h) > 4 

i=\ i>k 1 C[ - l> > 

Then by (W.b) there are < 5 < 1 and c > such that 

oo 

Pr{^(X) > cc 1/(1+5) (n)} < oo. 

n=l 

But by regular variation, there exists r > such that h~ r > cc 1 ^ 1+ ^ (n), and 
therefore 

oo 

X>{\P(X)>/>7}<oo. 

n=l 

Now by (W.c), for n large enough, there exists C < oo such that 

Pr{^(X) > Cc(n)} < p r j cinf N<IM^(^-fr) > 2 1 + (x) > 

I c(n) J 

Therefore, 

oo 

Pr{^(X) >Cc(n)} <oo. 

n=l ^ 

We are now prepared to give an integral test for a.s. convergence of the 
sequence (2.1). Notice the difference with the tightness criterion, which is 
due to the fact that, by Lemma 2.5, we have 

(2.23) either lim max *t*> = a., or limsup max ^ = 00. 

rwool<i< n \ n n^oo l<i<n \ n 

Theorem 2.7. Assume the usual hypotheses, with condition (WD.a)fl 
holding for some j3 G (0, 1/2), and, moreover, that either Bf = R rf or K(0) = 

\\K\loc. Set \(t) = V thf\ log ht\, as before. Then, eith 



er 



(2.24) lim J n/t " ||/ w - Ef n \\ %O0 = ||^|| 2 ||/ 1/2 |k,oo a.s. 
n^oo y 2\ log/t^l 



or 



/ nh d 

(2.25) limsup W — — ^-jyll/n - Ef n \\ 9 oo = oo a.s., 

n^oo y 2| logn^| 

according to whether 

/oo poo 
Pr{^(X) > X t }dt <oo or J Pr{#(X) > A t } dt = oo. 
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Proof. By Proposition 2.6, since A n is regularly varying, if the integral 
in (2.26) is infinite, then (2.25) holds. So, we must prove that 

POO 

(2.27) J Pv{^(X)>c\ t }dt<oo 

for all c > implies (2.24). We proceed as in the proof of Theorem 2.1, with 
the addition of the usual blocking and replacing, in the estimation of the sup 
over C n , the moment bound by an exponential inequality. By (2.4), we only 
have to consider the sup of our statistics over A n , B n and C n , the three sets 
defined as in the proof of Theorem 2.1, but with c n = 1 (and e n = 1/logn as 
before), and we can ignore the centerings on A n and B n . By monotonicity 
of h n and A n , we have 

^{t)YJUK{{x t -t)/h n ) 

max sup 

"-- ru Slip " . 

*GA 2fe A 2 fc 
Hence, we have, as before, 



Pr<^ max sup — — — — - — > e 

^<n<2k+^ t&An A n 



<e" 1 2 fc+1 Pr|^(X) > 



x l//3 



for all k large enough and some c > 0. But, by (2.27), this is the general 
term of a convergent series, thus proving that 

(2.28) lim sup V 1 " = a.s. 

n^oo teAn \ n 

Regarding B n (with c n = 1 and e n \ 0), we first note that, by regular 
variation, 

2k+ - ( / 1-/3 /^l log 1/2 



(J B n CB 2k :=\t:f(t)y(t)<c'ell 

n=2" 1 V 1 n 2 k 

v&(t)<c / 4(2 fe 4|log/i 2fc |) 1 / 2 | 
for some d > 1. Then, as in (2.11), 

mM » (t) st.A-HX, -«)/M 

2 k <n<2 k + 1 t £B n A n 
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Xok 



C*(Xj) , C*(XjKiEl<t<n,< ? feiJ(l^i--Xj-|<^: 

< max — ; — - — h max 



l<j<2 k + 1 X 2 k 



l<j<2 k + 1 



X 



2 k 



where is denned as before but with n = 2 k+1 , and c may be different from 
the constant in (2.11). Now, the maximum term tends to zero a.s. by (2.7) 
and Lemma 2.5, and the remainder term satisfies 



Pr 



max 

i<i<2 fe+1 



c\P(*;)/;£ 



3) 1 3 Z^l<i<n,i^j 



;K\Xi 



Xj I < h 2 k ] 



X 



>e < 



2 k 



c 



e2 



ak 



for some a > and all k large enough, as in the proof of Theorem 2.1. 
Therefore, 



(2.29) h ^'WS,^-W =l 



a.s. 



In order to control the sup of our statistics over C n — C n a fas defined 
in (2.14)], we will use Talagrand's exponential inequality [Talagrand (1994, 
1996)] in conjunction with the bound on the expected value of the sup of 
an empirical process given in (2.16). In a ready to use form for the problem 
at hand, it is as follows [Gine and Guillou (2001), equation (2.12)]: under 
assumption (2.15) above, and with the notation of (2.17) above, assuming 
further that 



< a < U/2 and yjna > £A/lo{ 



U 



there exist constants C and L such that, for all s > C, 

n 

£(/(&) 



(2.30) Pr 
where 



i=l 



> say/n\ /log — ><Lexp 



a 



D(s) := slog(l + s/4L) — > oo as s ^ oo. 



D(s) U 
——log — 
L a 



We apply this inequality to the class T n defined on the last part of the 
proof of Theorem 2.1, with U = U n and a = a n as defined there, so that 
log ^ x log re. Since, for a fixed and n large enough, a n — > 0, U n — > oo and 

\fna n j (U n Jlog — ) — > oo, the above applies to give that there exists C < oo 
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such that, for all a > and for all n large enough (depending on a), 
*(*) Ei=i(K((Xi - t)/h n ) - EK((X - t)/h n )) 



(2.31) 



Hence, 



Vx\ sup 



A, 



>C||/ 1/2 ^| 



D?.nB 



[Z.OZ) ri— >oo teCn, 



< Lexp{— 21ogn}. 

*(*) YZ=i{K({Xi - t)/h n ) - EK{(X - t)/h n )) 



Xn 



< C\\f 1 lH\\ D ^ Bf a.s. 



Combining (1.6), (2.28), (2.29) and (2.32), and letting a — > oo, we obtain 
the limit (2.24). □ 

We conclude this section with a few examples. We take ^(t) = f~ /3 (t). 
Other choices of are of course possible. 

Example 2.8. Suppose / :R^ (0, M] is continuous and 

f( x ) = c 1 e- c *W r 

for all |x| large enough, for some r > and for some constants c\ and ci- 
Then, / satisfies (D.a)-(D.c). Take 

h n = n~ a , 0<a<l. 

For simplicity assume c\ = ci = 1. It is easy to see that 

Pr{\X\ > u}xn^ r e^ r . 

Hence, 



Pr 



{/(*) 



>t 



(l-a)/(2/3) 



(logi 



,1/(2/3) 



1 



t (l-a)/(2/?)(l ogt )l/(2/?)-(l-r)/r ' 



Then the above theorems imply the following. For r > 1, which includes the 
symmetric exponential and the normal densities, the conclusion is that the 
sequence (2.1) with ^(t) = / _/3 (i) is tight (stochastically bounded) if and 
only if 



2(3 <1- a 



and that, if this is the case, then 
' n 1 -" \fn-Ef n 



(2.33) 



2a log n 



\K\ 



1/2-/3 



a.s. 
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The same is true for exponential densities if we replace in (2.33) the sup over 
R by the sup over R + . For < r < 1, if 2/9 < 1 - a, then the limit (2.33) 
holds. If 2/3 = 1 — a, different behaviors arise; namely, if (1 — r)/r > 1/(2/3), 
then the sequence (2.1) is not stochastically bounded; if (1 — r)/r = 1/(2/?), 
the sequence converges in distribution to the limit in distribution of the 
random variables 



\K\ 



max 



y/2an 2fi logn/^X;) 



VflLKI 



1/2-/91 



which is unbounded and can be easily computed (see the next example); 
if (1 — r)/r — 1/(2/3) < 0, we have convergence in probability in (2.33), but 
convergence a.s. holds only if (1 — r)/r — 1/(2/3) < — 1. 



Example 2.9. Suppose now the real density / is strictly positive, con- 
tinuous and 



/(*) 



c 

\x\i 



for all | a; | large enough, for some r > 1 and for some constant c. These 
densities also satisfy (D.a)-(D.c). Take h n = n~ a , a € (0, 1) as above. Then, 
(2.1) [again, with *(£) = / _/3 (£)] is tight if and only if 



0< 



1 1 -a 



r 2 ' 

and, if this is the case, then (2.33) holds true. 

Example 2.10. Let now f(x) = be the symmetric exponential 

density on R. Then, 



1 



Pr<( max 

l<i<nf(Xi) 



so that 



> u 



max 




if u>2, 
otherwise, 



l<i<n nPfP{Xi 
2/t 



where Z has distribution 

Pr{Z < t} -- 
Hence, if we take f3 E (0, 1/2) and 

h n = 



e 
0. 



z\ 



if t>0, 
otherwise. 



n 1 2 @ log n ' 
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nh r 



2 1 log h n \ 



fn — Ef n 



max 



.( \ K \ 



\K\ 



VV2(l-2/?) '2V2-/3 



The next two examples show that the above results are not true in general 
without conditions of the type of (D.b), (D.c) [and (W.b), (W.c)]. The first 
addresses smoothness and the second the existence of zeros of / on the 
closure of Bf. 

Example 2.11. It is easy to see that the double exponential density 
still satisfies conditions (D.a)-(D.c) and, hence, Theorems 2.1 and 2.7, but 
the density 

f(t):=ce- ee \ t>0, 

does not. Specifically, condition (D.b) fails for this density and we show 
below that, for all (3 G (0, 1) and for h n = n~ a , 

fn(t)-Ef n (t) 



(2.34) 



2 1 log h 



■ sup 

71 1 i>0 



oo a.s. 



Indeed, if K is continuous and strictly positive at the point t = —1/4, then 

Ef n (t) 



= cexp{/3e e }/ K (u) f (h n u + t) du 

7-1/2 



1/2 

> ci exp{/3e e * - e 

= ci exp{e e * \p — e et( - e 
Let t„ :=logn. Then, for large n, 



/ nh n Ef n (t n ) 
V2|log/ in | fP(t n ) 

On the other hand, 



> ciexp{e n [/3 - e 



n(e 



-1) 



]} > ci exp<^ (3 



Pr{fn(t n ) + 0} < Pr max X, > t n 

l<i<n A 



n 



which implies that 



/ nhn fnjtn) 

V2|log/ ln |// 3 (y 
and therefore (2.34) holds. 



a.s., 
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Example 2.12. This example shows that if the density / has a zero in 
R, then the asymptotic behavior of 



nh n 



log h n | 



fn — Efn 



depends on the local behavior of / at the zero point and is no longer con- 
trolled only by condition (2.2). Note that in this case condition (D.b) fails. 



For simplicity, assume that h n 



n 



(with a < 1) and K = L. 



■1/2,1/2] 



. Let 



/ be a density continuous on a neighborhood of and such that /(0) = 
and, moreover, for some s > 0, 

/(£)x|t| s ast^O. 

In particular, we assume that / is s times continuously differentiable at 
(for an even integer number s) and /^(0) = for j < s, f( s \0) > 0. It is 
easy to see that 

(2.35) Pr{|X| < t} asi^O. 

We will show that if s > ^ — 1, then, for all C > 0, 



(2.36) 



Pr< 



nh r 



2 1 log h ri 



fn — Ef n 



>c 



1. 



B, 



The proof is almost the same as in the previous example. Let t n — > be 
chosen in such a way that f(t n ) = e~ n . Note that t n = o(h n ). Then, using 
(2.35), we get 



' nhn Ef n {t n ) 
2\\ogh n \ fP(t n ) 




e^ n n a EK{n a {X - t n )) 



Pr t n 



n 



<X<t n + 



n 



e Pn n -{s+l) a 



OO. 



On the other hand, also using (2.35), if s > — — 1, then 

Pr{/n(*n) + 0} < Pr{3 i, 1 < i < n : X { G (t n - h n /2, t n + h n /2)} 
< n Vr{X £ {t n - h n /2, t n + h n /2)} 



nh 



s+l 



n 



l-a(s+l) 



0. 



This immediately implies (2.36). Now let f(t) = c\t\ s for \t\ < a and f(t) = 
otherwise. Then it is easy to check that condition (2.2) holds if and only if 



l-a, 



I + Thus, for large enough s, this condition does not imply the 



stochastic boundedness of (2.1) 
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3. Large normings. By Proposition 1.1, the central part of the process 
^(t)(fn(t) — Ef n (t)), that is, its sup over D a , for all a > 0, has an influence 
on the asymptotic size in probability of the sequence (2.1) and completely 
determines its a.s. limit. But if we normalize by a sequence larger than 
V nh^ | log h n | , this central part of the sup vanishes for all a > 0, and only 
the extremes of the range of i's should have an influence on the limit. This 
is what we examine in this section. As in the previous section, we will only 
consider regularly varying window sizes and normings. As is to be expected, 
the only possible limit a.s. in this situation is zero, and the sum is asymp- 
totically equivalent, in probability, to the maximum term. This is roughly 
the content of the following two theorems. 

Theorem 3.1. Assume the usual hypotheses, with condition (WD. a) p 
holding for some (5 E (0, 1], and, moreover, that either Bf = R rf or K(0) = 
1 1 if | |oo. Let dt be a strictly increasing regularly varying function such that 
dt/^t — > oo and dt > Ctr for some C > 0. Then, the sequence 

(31) r Y2=i{mxi-t)/h n )-EK{{x-t)/h n )) 



d n 

is stochastically bounded if and only if 

(3.2) limsupiPr{#(X) > d t } < oo. 



t— >oo 



Moreover, if condition (3.2) holds, then 

Y2=i{K{{Xi-t)/h n )-EK{{X 



(3.3) 



t)/K)) 



\K\ 



max ■ 

Ki<n 



dn 



oo 



dn 







in pr. 



Proof. The proof is similar to that of Theorem 2.1. First we consider 
(3 < 1. Necessity of condition (3.2) follows exactly in the same way. Here we 
indicate the few changes that should be made to the proof of Theorem 2.1 
in order to prove that (3.2) implies (3.1) and (3.3). First, and this is by far 
the main difference with Theorem 2.1, the sup of 

(3 4) *(*) Zti(K((Xi ~ t)/h n ) - EK((X - t)/h n )) 



d„ 

over D a , tends to zero a.s. for all a < oo by Proposition 1.1. Regarding the 
centering, consider the bound 

n^>{t)EK{{X - t)/h n ) 
dnPit) 

< c -^*(t)f(t)i(f(t) > h n ) + £^*(t)/i-'(t)i(/(t) < hn ), 



d,, 



d„ 
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where t £ Bf, 1 — > 5 and r is such that raft^ 1 ^ ^ /d n — ► 0, which is 
obtained as in the proof of Theorem 2.1. If the exponent of regular variation 
of nhn is strictly smaller than that of d n , then, since is bounded for 
all rj> (3, the sup over t £ Bf of this bound tends to zero and therefore we 
can simply ignore the centerings in (3.1) and (3.3). Otherwise, the second 
summand tends to zero uniformly in t E Bf and the first tends to zero 
uniformly on all t 6 Bf such that 

/(*)*(*)< 

raft* 

for any e n — > 0. So we can ignore the centerings for these values of t. As 
before, we take e n = 1/logra. 

Continuing in analogy with the proof of Theorem 2.1, we now define 

A n = {t£Bf.y(t)>4d n } 

with c n = (dniogn/dn) 1 ^ — > oo, and we get, as in (2.8) but now using the 
properties of d n , that 

lim sup — — - — — — — = m pr. 

(for < /? < 1). 
Next we set 

B n := \teB } : f(t)V(t) < 4"^, < <&k} 

in analogy with (2.9). Then, proceeding as in the proof of (2.10) with the 

only formal change of replacing A n by d n and V nh^/\ log h n \ by nh^/dn, we 
arrive at analogous conclusions, namely that the sequence 

sup 

teB n On 

is stochastically bounded and that in fact it can be represented as 

max — hOp(l). 

l<j<n a n 

(This requires using the properties of d n and h n but, given that proof, the 
details are straightforward.) 
Finally, we consider 

C n = B f \ (An U B n ) ={t€Bf. /(*)* (t) > 4^dn/(^)}. 

Using as before that >L implies, by (WD.a)^, that ^ < L _/3 / (1 ~^, we 
can take 



d n 
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We will consider two cases. 

If the exponent of regular variation of nh^ is strictly smaller than that 
of d n , then e n ~^d n j(nh'^) — > oo and therefore, since, by (WD.a)^, ||/^||oo < 
II/IIocT' 3 < oo, C n is eventually the empty set. 

Assume now that the exponent of regular variation of d n does not exceed 
that of nh n . Then e n ~^d n /{nhn) is eventually dominated by for any 
5 > 0, so that we eventually have f(t) > h r n and ^(t) < h~ r for some r > 
and all t G C n . So, we can apply (D.c) and (W.c), which, together with 
(WD.a)/?, immediately imply that we can take o n as follows: 



{CKhiWfW 1 - 2 ?, if /3 < 1/2, 

\ C K h*^ 2p - 1 \nh*/d n )W-mi-P) 1 if p > 1/2. 



Since U n is either slowly varying or tends to infinity and o n tends to zero 
as a negative power of n for (3 < 1/2, we get, in this case, that, eventually, 

< o n < U n /2 and log — x log n. 

The same conclusion holds for f3 > 1/2 since hn" 2 decreases as a negative 
power of n and the exponent of nh^/dn in the expression for a n is smaller 
than its exponent in the expression for U n . It is also easy to see, using 
^n/d n — > in the case < 1/2 and d n > CvP when (3 = 1/2 or (3 > 1/2, 
that, eventually, 

Vn<Jn > U n Vlog(U n /cr n ) x U n V\ogn. 
Then inequality (2.16) gives that 

T2=i(K((Xi - t)/h n ) - EK((X - t)/h n )) 



E ( sup 



< 



Cy/n logn o n 



for some C < oo independent of n, as long as n is large enough. For [3 < 1/2, 
this bound is, up to a multiplicative constant, of the order of 

d n 

and for (3> 1/2, it is of the order of 

e-^- 1 V 2 v^(n' 3 /d„) 1 /(2(i-/3)) ^7(2(1-/3)) ^ ^ 

since d n > Cn^ for some C > 0, and h n — > at least as a negative power of 
n. This completes the proof of the theorem for (3 < 1. 

For (3 = 1, since d n >Cn and H^/H^oo < 1 ; w e can ignore the centering 
for all t. Then we decompose Bf into A n defined as above and B n := {t G 
B f :V(t) < c^d n }. The proof of (2.8) and (2.10) with A n replaced by d n 
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follows as in the proof of Theorem 2.1, even with some simplification as B n 
is now a simpler set. □ 

We have assumed d n > Cn@ and < 1 in the above theorem. Next we 
show that these two assumptions are optimal. 

Remark 3.2. Take ^ = f~@. For the sequence (3.1) to be stochastically 
bounded, it is necessary, by the first part of Theorem 2.1, that the sequence 
{maxi<j< n (d n / /3 (Xj))~ 1 } be stochastically bounded, hence, by regular vari- 
ation of dt, that 



supnPrj-^y>d^} 



< oo. 



But if Bf = R rf , then condition (D.c) implies, as in the proof of Corollary 
2.2, that 

nPr I _L_ >d 1 je\>c- n 



f(X) - » J - d VP 

for all n and some c > independent of n. Hence, if *&(t) is of the order of 
f~P(t), then we must have d n > Cn 13 in Theorem 3.1. 

Remark 3.3. Suppose we take /3 > 1 in Theorem 3.1, and, again, let 
us take \l/ = f~@ . Then, we still have that (3.2) is necessary for stochastic 
boundedness of the sequence (3.1). But then (3.2) implies that 

Inn sup =0 m pr. 

n ^°°teA n d n fP(t) 

as before. On the other hand, if Bj = H d , then the set A n contains t's with 
f(t) arbitrarily small, and therefore, by (D.b), for some < 6 < /3 — 1, 



nEK((X -t)/h n ) . nh n 

sup a teu\ ~ sup a t3-i-sn\ 

teA n d n fP{t) teA n d n fP 1 d {t) 



Hence, the sequence (3.1) is not stochastically bounded, which is a contra- 
diction. So, Theorem 3.1 is not true for f3 > 1. 

The next theorem describes the almost sure behavior of ||/ n — Ef n \\^ ^ 
for large normings. 



Theorem 3.4. Assume the usual hypotheses, with condition (WD.a)^ 
holding for some (3 E (0, 1], and, moreover, that either Bf = R rf or K(0) = 
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Halloo. Let dt be a strictly increasing regularly varying function satisfying 
that lim^oo dt/Xt = oo and dt > Ct@ for some C > 0. Then, either 



(3.5) 



or 



lim 

n— >oo 



J2ti(K((Xi - t)/h n ) - EK((X - t)/h n )) 



d,, 







a.s. 



*, oo 



(3.6) lim sup 



according to whether 



£? =1 (K(PQ - t)/h n ) - EK{(X - t)/h n )) 



dm 



OO, 



*,oo 



(3.7) 



Pr{^(A") > d t }dt < oo or 



Pr{^(X) > dt}dt = oo. 



Proof. Necessity and the part of sufficiency dealing with the sets A n 
and B n follow by a straightforward combination of the proofs of Theorems 
2.7 and 3.1. The only difference with previous proofs is in the estimation of 
the supremum of the processes over the sets 

C n = {t G B f : /(i)tt(t) > s^d n l{nhi)-\. 

Here, as in the corresponding part of the proof of Theorem 2.6, we use Tala- 
grand's inequality. However, d n is large and it may fall out of the "Gaussian 
range" of the inequality. With the notation put forward above, and with the 
assumptions 



<a n <U n /2 and ^/na n > U n \ log 



0",, 



shown to hold for all n large enough in the previous proof, Talagrand's 
inequality in the version from Gine and Guillou [(2001), Proposition 2.2] 
gives 



Pr< sup 
(3.8) l teC " 

< Lexp 



Xi-t 



EK 



(X-t 



hn J V 

1 ed n ( ed n U n 
log 1 + 



> ed n 



(I), 



for some L that depends only on A and v [from (2.15)], and for all n large 
enough, as long as 

ed n >(J 



^na n y/\ogU n /a r , 

for a certain constant C < oo. This last condition is eventually satisfied by 
all e > since log(U n /a n ) x logn and d n j {^/n log na n ) — > oo, as can be easily 
seen directly from the definitions and properties of these quantities. 
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Now, by the hypotheses on h n and d n , and since e n = 1/logn, there exists 
5 > such that 



ed r . 



(3.9) j^=e-e 
If l/2<0< 1, then 



d \V(i-i9) 



h -df>/{i-fi > Ce-dK; d f ) K 1 -®>n 5 . 



log 1 + 



Lnal 



log 1 + 



>log logn. 



If /?< 1/2, 



log 1 + 



sd n U n 
Lnal 



log 1 + 



j N (1-2^/(1-/3) 



Le£ 



which is of the order of log ri if the exponent of regular variation of d n is 
strictly larger than that of n/i^, and satisfies 



lim n log 1 + 



A \ (l-2/3)/(l-/3) 



oo 



for all 5 > if the exponents of d n and n/i^ coincide. (This can be readily 
seen using the properties of regular variation and that log(l + r) ~ r for 
r small.) Combining the last three estimates with the bound (3.9), we get 
that, for the cases considered, 



(3.10) 



(/) < exp(-n 5 ) 



for some 5 > 0. Finally, if (3 < 1/2 and the exponent of variation of d n is 
smaller than the exponent of nh^, then 



log 1 + 



ed n U n \ ^ ed ri U n 

and we have, for constants L independent of n (as long as n is large enough) 
and that vary on each occurrence, 



(3.11) 



(I) < Lexp 
= Lexp 



lift 

L nal 



Lexp 



lift 

L nht 



L VA r 



\ogh n \ ) < Lexp(— Mlogn), 



where M can be made as large as we wish, as long as we take n large 
enough. (Here we have used d n /X n — > oo and | \ogh n \ x logn.) This covers 
all the cases, and we obtain, combining (3.8), (3.10) and (3.11), that 



r^ sup 

t&Cn 



i=l 



Xi-t 



EK 



( K )) 



> ed n > < oo 
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lim sup 



i=i 



Xj-t 

hr, 



EK 



( K )) 



a.s. 



This completes the proof of the theorem. □ 



The results in this section obviously apply to the densities in Exam- 
ples 2.8-2.11. For instance, let / be the symmetric exponential density on 
R considered in Examples 2.8 and 2.10, and let h n = n~ a , < a < 1. Then, 
Theorem 3.1 shows that 

fn — Ef n 



n 



l-a-P 



I K 1 1 ,v, Z^ , 



where Z is the random variable defined in Example 2.10, if and only if 

1 — a 

— </?<i; 

and Theorem 3.4 shows that, for c(i) strictly increasing and regularly vary- 
ing, 

f n -Ef n 



n 



a.s. 



if and only if 



dt 
W) 



< oo. 



A similar statement holds true for normal densities. 

Remark 3.5. Suppose that K is a uniformly bounded class of kernels 
supported by a fixed bounded set and such that the class 



K 



h 



: t E R , h>0, KelC 



is measurable and has covering numbers 



N(F,L 2 (P),\\K\\ L2{P) e)< {- 



0<e< 1, 



for some A and v finite and positive and for all probability measures P. [In 
particular, KL may be a subset of the linear span of a finite set of functions 
k as defined in condition (K)]. Suppose we wish to consider 

fn — Ef n 



sup 

Keic 



oo 
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where c n is d n or A n , as defined above. Then uniform boundedness and 
uniformity of the support allow us to deal with the sup over A n and B n , and 
the entropy bound, with the sup over C n , just as in the previous theorems. 
The sup over the central part D a is handled in Mason (2004). So, it is 
straightforward to prove a uniform in K E K version of our results. It is 
also possible to prove a functional law of the logarithm in our setting by 
following Mason (2004). 
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